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Holographic Apparatus and Method 

The present invention relates to a method of displaying a holographic ally generated 
video image composed of plural video frames, and to apparatus constructed and 
5 arranged to display a holographically generated video image having plural video 
frames. 

For the sake of clarity, the term "video data" is not limited to data representing any 
particular type of image content and is intended to include data that is representative of 
10 one or more still images, wherein the data are periodically or regularly refreshed. 

Holographically generated 2D video images are known to possess significant 
advantages over their conventionally projected counterparts, especially in terms of 
definition and efficiency. However, the computational complexity of the current 
15 hologram generation algorithms preclude their use in real-time applications. 
Furthermore, even if the existing algorithms were fast enough, the images generated 
by them are not of sufficient quality for use in a display application. 

Traditionally, hologram generation algorithms may proceed by finding holograms that 
20 minimise the mean-squared error (MSE) between the desired image and the image 
produced (the replay field, or RPF) when the hologram is illuminated with coherent 
light. The inventors have shown that human subjects consider lower signal and noise 
variance in the RPF to be far more important in display applications. Using this result, 
a new algorithm optimised to the psychometrically-determined perceptual measure of 
25 image quality has been developed. 

Holographically generated images may be formed by light leaving a device exhibiting 
a phase-modulating effect upon light incident upon the device. Typically the device is 
a spatial light modulator (SLM) and has a field of addressable phase-modulating 
30 elements. In some SLMs the phase-modulating elements are a linear or one- 
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dimensional array of elements; in others a two dimensional array may be provided. For 
simplicity many SLMs have a regular 2-D array of like, generally square, phase- 
modulating elements; it is however not necessary for the phase-modulating elements 
to be alike in size or shape. 

5 

In general the phase-modulating elements of SLMs are often referred to as "pixels", 
and the elements themselves as "pixellated". The terms are used interchangeably 
herein. It is not intended herein, when the term "pixel" is used, that any directly 
viewable picture content is to be seen on the elements concerned. 

10 

An object of some embodiments of the invention is to achieve high quality real-time 
holographic video projection. 

An object of some embodiments of the invention is to provide an algorithm for 
15 forming a CGH, which algorithm involves a low number of computationally intensive 
steps. 

An object of some embodiments of the invention is to provide a fast algorithm for 
forming a CGH, which algorithm can be implemented on relatively unsophisticated 
20 hardware, such as is currently commercially available. 

According to one aspect of the invention there is provided a method of displaying a 
holographically-generated video image composed of plural video frames, the method 
comprising providing for each frame period a respective sequential plurality of 
25 holograms and displaying the holograms of the plural video frames for viewing the 
replay field thereof, whereby the noise variance of each frame is perceived as 
attenuated by averaging across said plurality. 

In an embodiment the providing step comprises generating each hologram by 
30 implementing an algorithm having a single computationally intensive step. 
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In an embodiment the single step is a Fourier transform step. 

In an embodiment the algorithm is arranged, for each said plurality of pixellated 
5 holograms, to a) form first data having amplitude equal to the amplitude of the desired 
pixel and identically distributed uniformly random phase; b) inverse Fourier transform 
the first data to provide second data; c) shift the second data in the real direction in the 
complex plane sufficiently to form a third data set in which the phase of each data 
point is small; d) form as a fourth data set the magnitude of the third data set; and e) 
10 binarise the fourth data set to form a fifth data set for display as a said hologram. 

In an embodiment the step of binarisation comprises thresholding about the median of 
the fourth data set whereby the fifth data set has dc balance and low reconstruction 
error. 

15 

In another embodiment the step of binarisation comprises thresholding the fourth data 
set about zero. 

In an embodiment the display device comprises a spatial light modulator having a 
20 pixellated phase mask imposing phase shifts of substantially 0 and 7i/2, wherein the 
effect of the display device showing a binary hologram in co-operation with the two- 
phase phase mask simulates a four-phase modulation, wherein each pixel has one of 
the values [1J,-1,-J]. Other phase shifts would be possible provided a difference of 
substantially n/2 or n nil is present between the two levels. 

25 

In an embodiment the pixel values of the phase mask are spatially random. 

In another embodiment, the pixel values of the phase mask are decided 
deterministically. 

30 
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In an embodiment step b) comprises inverse Fourier transforming and applying the 
phase mask values to the first data to provide the second data. 

An alternative family of embodiments uses a Fresnel transformation instead of a 
5 Fourier transform. 

Some embodiments are restricted to 2D holograms; others provide 3D. 

According to another aspect of the invention there is provided apparatus constructed 
10 and arranged to display a holographically-generated video image having plural video 
frames, the apparatus having processing means arranged to provide for each frame 
period a respective sequential plurality of holograms and a display device arranged to 
receive the sequential plurality of holograms of each frame and to display the 
holograms of the plural video frames of the video image for viewing the replay field 
15 thereof, whereby the noise variance of each frame is perceived as attenuated by 
averaging across said plurality of holograms. 

In an embodiment the processing means is arranged to generate each said hologram by 
implementing an algorithm having a single computationally intensive step. 

20 

In an embodiment the single step is a Fourier transform step. 

In an embodiment the algorithm is arranged, for each said plurality of pixellated 
holograms, to a) form first data having amplitude equal to the amplitude of the desired 

25 image pixels and identically distributed uniformly random phase; b) inverse Fourier 
transform the first data to provide second data; c) shift the second data in the real 
direction in the complex plane sufficiently to form a third data set in which the phase 
of each data point is small; d) form as a fourth data set the magnitude of the third data 
set; and e)binarise the fourth data set to form a fifth data set for display as a said 

30 hologram. 
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In an embodiment the step of binarisation comprises thresholding about the median of 
the fourth data set whereby the fifth data set has dc balance and low reconstruction 
error. 

5 In another embodiment, the thresholding of the fourth data set is performed about 
zero. 

In an embodiment the display device comprises a spatial light modulator having a 
pixellated phase mask imposing phase shifts of substantially 0 and 7t/2, wherein the 
10 effect of the display device showing a binary hologram in co-operation with the two- 
phase phase mask simulates a four-phase modulation, wherein each pixel has one of 
the values Other phase shifts would be possible provided a difference of 

substantially tz/2 or n n/2 is present between the two levels. 

15 In an embodiment the pixel values of the phase mask are spatially random. 

In an embodiment step b) comprises inverse Fourier transforming and applying the 
phase mask values to the first data to provide the second data. 

20 Embodiments of the hologram generation algorithm / method can be run sufficiently 
fast to provide a number, e.g. 40, holograms per video frame, where a video frame is 
1/25 second. It has been found that there is a statistical result that the noise variance of 
the time average of N images, each with independent noise of some arbitrary 
distribution, falls as UN, Embodiments of a first algorithm embodying the invention 

25 can therefore generate, in real time, a set of N holograms per video frame 
('subframes') to produce RPFs with independent noise. If the frames are displayed 
quickly enough then the limited temporal bandwidth of the eye is exploited to impart 
the effect of noise variance attenuation due to averaging. Embodiments of a second 
algorithm can produce 2N holograms from N Fourier steps. 

30 
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According to a further aspect of the invention there is provided a method of providing 
a pixellated hologram comprising forming first data having amplitude equal to the 
amplitude of the desired image pixels; inverse Fourier transforming the first data to 
provide second data; shifting the second data in the real direction in the complex plane 
5 sufficiently to form a third data set in which the phase of each data point is small; 
forming as a fourth data set the magnitude of the third data set; and binarising the 
fourth data set to form a fifth data set for display as said hologram. 

According to a yet further aspect of the invention there is provided a method of 
10 providing a pixellated hologram comprising forming first data having amplitude equal 
to the amplitude of the desired image pixels; inverse Fresnel transforming the first data 
to provide second data; shifting the second data in the real direction in the complex 
plane sufficiently to form a third data set in which the phase of each data point is 
small; forming as a fourth data set the magnitude of the third data set; and binarising 
15 the fourth data set to form a fifth data set for display as said hologram. 

According to still another aspect of the invention there is provided a method of 
generating a pixellated holograms, the method comprising forming a first data set, the 
members of said first data set having respective amplitudes equal to the amplitudes of 
20 respective desired pixels; performing an inverse Fourier transform on the first data set 
to provide second data set; shifting the second data set in the real direction in the 
complex plane sufficiently to form a third data set in which the phase of each data 
point is small; forming as a fourth data set the magnitude of the third data set; and 
binarising the fourth data set to form a fifth data set for display as a said hologram. 

25 

The forming step may comprise forming data having identically distributed random 
phase. 

The step of binarisation may comprise thresholding about the median of the fourth 
30 data set whereby the fifth data set has dc balance and low reconstruction error. 
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The display device may comprise a spatial light modulator having a pixellated phase 
mask imposing phase shifts of substantially 0 and 7t/2, wherein the effect of the display 
device showing a binary hologram in co-operation with the two-phase phase mask 
5 simulates a four-phase modulation, wherein each pixel has one of the values 

Other phase shifts would be possible provided a difference of substantially 7t/2 or n n/2 
is present between the two levels. 

The pixel values of the phase mask may be spatially random. 

10 

The method may further comprise applying the phase mask values to the first data to 
provide the second data. 

The image may be a 2 dimensional image. Alternatively the image may be three 
15 dimensional. 

Exemplary embodiments of the invention will now be described with reference to the 
accompanying drawings in which 

Figure 1 shows a block schematic drawing of apparatus embodying the 
20 invention; 

Figure 2a shows a conjugate image present due to binary SLM; 
Figure 2b shows the conjugate image eliminated with phase mask; 
Figure 2c shows the conjugate image eliminated using Fresnel holography 
Figure 3a shows an exemplary replay field produced by an embodiment of a 
25 method of the invention; 

Figure 3b shows a replay field similar to that of Figure 3a but produced by 
direct binary search. 

Figure 4 shows a partial cross-sectional view through an SLM having a pixellated 
30 phase mask imposing phase shifts of 0 and n/2. 
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Referring to Figure 1, a signal (2) representing an p x q pixel input intensity field 7^, 
generated directly from a video signal is input to a processing device (20) having a 
first processing block (1). The processing device (20) in this embodiment includes a 
5 field programmable gate array (FPGA) running code to enable it to perform the 
required functions. In other embodiments an ASIC is used and in yet others a 
programmed general purpose computer is used. The first processing block (1) for each 
said plurality of pixellated holograms, forms a first data set 7^ rt) at an output (3) by the 

expression = <^7^exp( j®™), so that the first data set has amplitude equal 

10 to the amplitude of the desired pixel and identically distributed (i.i.d.) uniformly 
random phase. 

The first data set (3) is applied to a second processing block (4) which forms a second 
data set G£° such that G£ } =F ~ l [T™] at output (5) where F 1 represents the 2D 
1 5 inverse Fourier transform. 

The second data set is then shifted by third processing block (6) in the real direction in 
the complex plane (i.e. to the right) sufficiently to form a third data set (7) in which 
the phase of each data point is small. The third processing block (6) forms R as the 
20 smallest positive real such that \G™\ < R Vjc, y,n . and adds a real a where a» R to 

each data item of the second set to form a third set a+ G^ } at output (7). 

The third data set at output (7) is applied to a magnitude-forming fifth processing 
block (8) performing the function = |or + G£°| such that is output as a 

25 fourth data set (9). 
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20 



The fourth data is then supplied to a binarisation stage (10) performing the function 



for display as a said hologram. 

The fifth data set (11) is supplied to a ferroelectric liquid crystal SLM (12) for display 
and viewing. Although a ferroelectric liquid crystal spatial light modulator is used in 
this embodiment other devices, for example including nematic SLMs, OASLMs 
(optically-addressed spatial light modulators), and also more exotic types of liquid 
crystal displays such as electroclinic, pi cell, flexoelectric, antiferroelectric, 
ferrielectric, V-shaped switching cells, and guest-host dye cells may be substituted, in 
addition to non-liquid crystal technologies such as OLED displays, vacuum 
fluorescent displays, electroluminescent displays, MEMS devices such as DMDs. The 
device may be transmissive or reflective. 

The presence of only a single computationally intensive step , i.e. the inverse Fourier 
transform, allows current hardware to generate in real time plural, e.g. 40, holograms 
per video frame. 

In a second embodiment, a modified process generates sets of 2N distinct p x q binary 
phase holograms , each of which produces a replay field that approximates the 

same target image. The key feature of the process is that the noise fields produced by 
each hologram are i.i.d., satisfying the conditions above. 

The process begins with the specification of a target intensity image T^, and proceeds 
as follows: 

1. Let T™ = ^/7^exp( where is uniformly distributed between 0 and 

2k for 1 < n < N, 1 < x < p, 1 < y < q 




where Q w = 



median(M^ } ) to form a fifth data set (11) 
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2. Let = F _1 [7^ n) ] where F _1 represents the 2D inverse Fourier transform 
operator, for \ <n<N 

3. Let Af = SiiG™ } for 1 < n < N 

4. Let AT £ +A ° = 3{G£ } } for 1 < n < N 

f-1 ifM in) <0 (n) 

5 5. Let H v=U ifM 7n)>g(«) where G (rt) =median(M<;>) and 

1 < n < 2A^ . 

Steps 3 and 5 of this process are exactly equivalent to steps 3, 4 and 5 of the first 
embodiment, but an additional step (number 4 in this process) is added. The 
10 holograms produced here are thus exactly the same as the holograms 

produced by the original algorithm, only here the holograms M ( ^ +N) are also provided 
"for free" -i.e. without a need for further Fourier transform step. 

Step 1 forms N targets 7^° equal to the amplitude of the supplied intensity target T , 
15 but with i.i.d. uniformly-random phase. Step 2 computes the N corresponding full 
complex Fourier transform holograms Steps 3, 4 compute the real part and 

imaginary part of the holograms, respectively. Binarisation of each of the real and 
imaginary parts of the holograms is then performed in step 5: thresholding around the 
median of ensures equal numbers of -1 and 1 points are present in the 

20 holograms, achieving DC balance (by definition) and also minimal reconstruction 
error. Hence, for one Fourier transform operation performed upon 7^, two binary 
phase holograms Uxy are provided as opposed to one hologram obtained using the 
process of the first embodiment. 
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In another embodiment, steps 3-5 perform not a binary quantisation but a quantisation 
to more than two phase levels. In the latter case, only one hologram is produced per 
Fourier transform performed upon T^. 

5 Any purely real hologram (such as a binary phase hologram employed in some 
embodiments) produces a conjugate image in the replay field, reducing the usable area 
and hence the efficiency by half. 

The conjugate image can be removed by generating a four-phase hologram where each 
10 pixel takes one of the values although such holograms cannot be displayed 

on an inherently binary device such as a ferroelectric SLM. More than two phase 
levels can be displayed on a nematic liquid crystal device, but commercially-available 
device of this type are currently too slow for high frame-rate applications, as provided 
by the algorithm described above. 

15 

However by etching a pixellated phase mask of pixel values 1 and j (thus imposing 
phase shifts of 0 and n/2 respectively) on the top surface of a binary-phase SLM the 
conjugate image can be removed. This is achieved by randomly setting the pixel 
values of the mask such that the net modulation imposed by the combination of the 

20 SLM and phase mask lie in the set [lj>l,-yL which provides the extra degree of 
freedom required to remove the conjugate image (at the expense of additional RPF 
noise), despite the fact that the SLM itself is binary. This technique has been used in 
correlator design and is here applied to the display application. Although the pixel 
values of the mask are random, and thus have "dc balance" in that there are 

25 statistically equal numbers of each value of phase shift, the mask itself is fixed and the 
pixel position of each value of phase shift is known. 

In other embodiments, a distribution of phase shifts that is not completely random is 
used. 

30 
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To modify the algorithm described above to use a phase mask, the second block (4) is 
modified to provide a such that 

Q(n) __ L *» J 
*y p 

where Pxy defines the phase mask, which has been generated randomly so that each 
5 pixel has an equal probability of taking the value 1 ory. 

Referring to Figure 4, a cross-sectional view through a part of a binary SLM (100) is 
shown. The SLM (100) has plural pixels (120-123) in the drawing although it will be 
understood that the pixels are in fact arranged in a two dimensional matrix. Each pixel 

10 (120-123) is capable of being electrically controlled so as to provide a phase shift of 0 
or 71. Each pixel (120-123) of the SLM has a respective phase-mask pixel (130-133) 
overlying it and disposed in correspondence with it. As will be seen in the Figure the 
first, third and fourth phase mask pixels (130, 132, 133) have a relatively low 
thickness whereas the second phase mask pixel (131) has a relatively high thickness. 

15 The thickness difference is selected on the basis of the material of the phase mask and 
on the basis of the wavelength of the light used. The selection is such that the light 
passing through the thicker pixels (131) is subject to a phase shift of n more than the 
light passing through the thinner pixels (130,132,133). 

20 Where the SLM or like device is reflective, the light will pass twice through the phase 
mask, and corresponding changes in thickness will be needed. 

In the present embodiment the phase mask is formed as a coating on the SLM (100). It 
would of course be possible to use other arrangements for example use two 
25 substantially identical SLMs with one providing a data display and the other forming 
the phase mask. 
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To evaluate the efficacy of this technique in simulation, holograms were generated for 
the two test patterns with each hologram being replicated twice, the replication step 
being performed between steps 2 and 3 in the algorithm. The replay fields formed 
demonstrated that the conjugate image is removed effectively. 

5 

The foregoing discussion relates to producing desired 2D structure in the far field 
using Fourier holograms. The method can also be extended to produce Fresnel 
holograms for the production of structure in the near field, which is particularly useful 
for a lensless optical configuration. Fresnel holography offers a further advantage of 
10 enabling lensless projection, since the hologram itself encodes the focussing element, 
giving an obvious weight, cost and size saving over conventional projective display 
systems. 



15 



20 



The discrete Fresnel transform is closely related to the Fourier transform, and 
describes the near field Txy produced at distance /by the hologram Gxy (of dimensions 
X by Y and corresponding pixel sizes A x and A^) when illuminated with coherent light 
of wavelength A When imaging not too close to the hologram the transform is given 
by 
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Fresnel holograms are generated that form the target image at the specified distance/ 
from the hologram. 

The techniques described may be modified for the generation of 3D holograms for 3D 
5 holographic video display. A 3D hologram of an object is simply a recording of the 
complex electromagnetic field (produced by light scattered by the object) at a plane in 
front of the object. By Huygens' principle, if the EM field distribution on a plane P is 
known, Huygens wavelets can be propagated through space to evaluate the field at any 
point in 3D space. As such, the plane hologram encodes all the information necessary 
10 to view the object from any position and angle in front of the plane and hence is, in 
theory, optically indistinguishable from the object. In practice, limitations in the pixel 
resolution of the recording medium restricts the viewing angle 0 which, as in the 2D 
case, varies inversely with the pixel size A. 



15 Consider a plane, perpendicular to the z-axis, intersecting the origin, and one point 
source emitter of wavelength JL and amplitude A at position (X, 7, Z) behind it. The 
field F present at position (x, y) on the plane - i.e. the hologram - is given by 



F(*,y)=^expj^rj W«hr = j{x-X) 2 +(y-Y) 2 +Z 2 



If a 3D scene is regarded as M sources of amplitude A, at (X„ F„ Z,), the linear nature 
20 of EM propagation results in the total field hologram F being 

F (*> y) = S^ ex p(^^) with n = ^x-Xf+iy-Ytf+Zf 

If y) is to be sampled over the region x min < x < x^ x , y^ n < y < ^ to form an m 
x m hologram F^, there is obtained: 

F *y -Z-JT^P 



with r t = k Xnin + x *™ ^= - X,. ) 2 + ( yniD + y y ™* -Yrf+Z? 
V m m 
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There is thus presented an algorithm (with an SLM phase mask) that generates N full- 
parallax 3D holograms for a given array of point sources, as above. 

Let F ^ =X!^ir ex pf + I with r < as above where *| n> is uniformly 

m W \ A J 

distributed between 0 and 2n for l<n<N,l<i<M 



Let = — 2L -, where is the precomputed [1J] phase mask as described in the 
previous section. 

Let R be the smallest positive real such that |g^ } |< R Vx,y,« . R exists since each 
10 value taken by is finite and so has compact support 

Let = |a + G™\ , where a is real and very much greater than R. 



i-l if M {n) <O in) 
Let H™ =\ „ » (n , where Q (n) = median(M^ } ) 



Alternatively, to obtain two holograms per Fourier transform then the preceding three 
15 steps are replaced by the following: 
Let M< n) = 9t{G£ ) } for 1 < n < N 

Let M ( x n y +N) = 3{G£> } for 1 < n < N 

f-1 if M in) <O ln) 

Let ff« -I . f m 7 b) ^ * where <2 <n) = median^ ™ ) and 1 < „ < 2N 



20 To test this algorithm, there is considered the calculation of TV = 8 holograms of 
resolution 512 x 512 and size 2mm x 2mm centred at the origin of plane P, giving a 
pixel size of A = 4|jim and hence a viewing angle of around 9 degrees under coherent 
red illumination (k= 632nm). The 3D scene used was a set of 944 point sources that 
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formed a wireframe cuboid of dimensions 12cm x 12cm x 18cm, located at a distance 
of 1.91m from the plane. 

The simulated RPFs produced were calculated by propagating Huygens wavelets from 
5 the TV holograms in turn through a pinhole aperture K onto a virtual screen (a plane 
perpendicular to the line from the centre of the cube to the pinhole), and recording the 
time-averaged intensity distribution on the screen. 

Unfortunately, in contrast with algorithm described for 2D, the computation of step 
10 one in the 3D algorithm above may not be realisable in real time with current 
consumer hardware for a scene with anything but a very small number of point 
sources. However, the required holograms generated by the algorithm can be 
computed and stored for recall in real time. Much more powerful computation may be 
needed for real-time three-dimensional hologram generation, although it has been 
15 found possible to optimise the calculations in a way that suggests that such 
computation may not be needed. 

The holographic projection method detailed above is realisable in hardware. At the 
time of writing, commercially available FPGAs (field programmable gate arrays) and 

20 other programmable hardware such as DSPs can be programmed using commercially 
available or custom code to compute 512 x 512 Fourier transforms at a rate of at least 
2400 frames/sec, sufficient for full-colour video (25 frames/sec, 3 colour planes, N = 
32). For the display device, a commercially available SLM can display a sufficient 
frame rate at at least 512 x 512 resolution. It is possible to use separate illumination 

25 devices where multiple colours are needed -for example for a full colour display. 
Alternatively a single device capable of multiple colour output can be used instead. 
Two significant design issues have been identified. 

Firstly, since optical loss in the phase holographic element is small and total energy is 
30 conserved, nearly all of the incident optical energy is routed into the RPF. Therefore, a 
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target RPF with fewer "on" points will appear brighter than one with many, assuming 
total energy is distributed approximately evenly between each "on" pixel in the RPF. 
As a result, a controller causes the intensity of the source to be modulated 
correspondingly to achieve a uniform overall brightness between frames, in 
5 accordance with the number of "on" points in each frame. Direct amplitude 
modulation of a laser is undesirable due to nonlinearity, and thus in an embodiment 
such amplitude modulation is replaced by pulse-width modulation within each 400 fxs 
pulse interval with a duty cycle proportional to the desired brightness to achieve the 
required average intensity. 

10 

Secondly, a design issue comes from the relationship between RPF size and 
illumination wavelength. This is overcome in one embodiment by a lens system 
corrected for chromatic aberration at the three wavelengths of the sources and in 
another by using Fresnel holography to build compensation for this effect directly into 
15 the holograms. 

The results of the approach of the invention show in simulation a RPF exhibiting a 
noise energy of two orders of magnitude lower than direct binary search (DBS), with 
calculation times six orders of magnitude faster. Experimental results are in agreement 
20 with the theory, and show a previously undemonstrated level of contrast and accuracy 
for a holographically generated image even with a test device that is a very old design 
of ferroelectric SLM. 

The invention has been described above in the context of binary operation and binary 
25 SLMs. However it will be understood that the algorithm itself is capable of extension 
to more than 2 phase levels. 

An alternative to step 5 set out above is to use error diffusion. As known to those 
skilled in the art, Error Diffusion is a way of converting a continuous function to a 
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quantised function, whilst ensuring that the noise resulting from this operation is 
moved to an area of the replay field which does not impinge upon the desired image. 

Although embodiments of the invention have been described, these are not limiting on 
5 the invention itself. This extends to the full scope of the appended claims. 



